Schema Change: Field deprecations/sunsetting#155
Conversation
| | ------------- | ----------- | --- | --- | ----- | | ||
| | `shopify__order_lines` | Removed columns | `variant_fulfillment_service`<br>`variant_grams`<br>`variant_weight`<br>`variant_weight_unit`<br>`variant_option_1`<br>`variant_option_2`<br>`variant_option_3` | | Completes deprecation of `product_variant` source, [which was removed in earlier Shopify connector updates](https://fivetran.com/docs/connectors/applications/shopify/changelog#january2025). | | ||
| | `shopify__inventory_levels` | Removed columns | `variant_fulfillment_service`<br>`variant_inventory_management`<br>`variant_grams`<br>`variant_weight`<br>`variant_weight_unit`<br>`variant_option_1`<br>`variant_option_2`<br>`variant_option_3` | | Columns sourced from deprecated `product_variant` source, [removed in earlier Shopify connector updates](https://fivetran.com/docs/connectors/applications/shopify/changelog#january2025). | | ||
| | `shopify__daily_shop`<br>`shopify_gql__daily_shop` | Data changes | Order metrics may be inflated | Order metrics now correctly calculated | Soft-deleted staging shop records filtered out, which previously could incorrectly inflate metrics. | |
There was a problem hiding this comment.
| | `shopify__daily_shop`<br>`shopify_gql__daily_shop` | Data changes | Order metrics may be inflated | Order metrics now correctly calculated | Soft-deleted staging shop records filtered out, which previously could incorrectly inflate metrics. | | |
| | `shopify__daily_shop`<br>`shopify_gql__daily_shop` | Data changes | Order metrics may be inflated | Order metrics now correctly calculated | Soft-deleted staging shop records are now filtered out, which previously could have incorrectly inflated metrics. | |
|
|
||
| ## Documentation | ||
| - Adds deprecation notes to fields no longer supported by the connector. These fields now return null and will be removed in a future release. | ||
|
|
There was a problem hiding this comment.
Should there be an under the hood entry about adjusting the horizontal integrity test?
There was a problem hiding this comment.
I'd say no, as we are trying to tighten up our changelogs to be more customer-focused and the customer won't really have interest in what our internal tests are.
| | `shopify__order_lines` | Removed columns | `variant_fulfillment_service`<br>`variant_grams`<br>`variant_weight`<br>`variant_weight_unit`<br>`variant_option_1`<br>`variant_option_2`<br>`variant_option_3` | | Completes deprecation of `product_variant` source, [which was removed in earlier Shopify connector updates](https://fivetran.com/docs/connectors/applications/shopify/changelog#january2025). | | ||
| | `shopify__inventory_levels` | Removed columns | `variant_fulfillment_service`<br>`variant_inventory_management`<br>`variant_grams`<br>`variant_weight`<br>`variant_weight_unit`<br>`variant_option_1`<br>`variant_option_2`<br>`variant_option_3` | | Columns sourced from deprecated `product_variant` source, [removed in earlier Shopify connector updates](https://fivetran.com/docs/connectors/applications/shopify/changelog#january2025). | | ||
| | `shopify__daily_shop`<br>`shopify_gql__daily_shop` | Data changes | Order metrics may be inflated | Order metrics now correctly calculated | Soft-deleted staging shop records filtered out, which previously could incorrectly inflate metrics. | | ||
|
|
There was a problem hiding this comment.
Add deprecated columns to table.
There was a problem hiding this comment.
Good callout! Adding that in.
| description: (DEPRECATED) The number of times that the redeem code has been used. This field now returns null following a connector update and will be removed in a future release. | ||
| - name: code | ||
| description: The redeemable code that a customer can enter at checkout (e.g., `SUMMER20`). | ||
| - name: created_by_description |
There was a problem hiding this comment.
Neither the body_html or published_scope field from the stg_shopify_gql__product model is documented in the yml, but they should probably be added and then documented as being deprecated. Same goes for pre_tax_price and pre_tax_price_set from the stg_shopify_gql__order_line model, inventory_quantity_management, inventory_quantity_adjustment, and presentment_prices fields from the stg_shopify_gql__product_variant model, and the cost field from the stg_shopify_gql__inventory_item.sql model.
There was a problem hiding this comment.
I think this is ok as is? These fields were only present into the REST models and never brought into the GraphQL models, so we don't need to add deprecation documentation as they aren't currently present anywhere in the GraphQL macro/staging flow.
Macros (present in REST, not in GraphQL)
https://github.com/fivetran/dbt_shopify/blob/main/macros/staging/get_order_line_columns.sql#L40
https://github.com/fivetran/dbt_shopify/blob/main/macros/staging/get_product_variant_columns.sql#L44
https://github.com/fivetran/dbt_shopify/blob/main/macros/staging/get_inventory_item_columns.sql#L35
Models (present in REST, not in GraphQL)
https://github.com/fivetran/dbt_shopify/blob/main/models/graphql/staging/stg_shopify_gql__order_line.sql
https://github.com/fivetran/dbt_shopify/blob/main/models/graphql/staging/stg_shopify_gql__product_variant.sql
https://github.com/fivetran/dbt_shopify/blob/main/models/graphql/staging/stg_shopify_gql__inventory_item.sql
There was a problem hiding this comment.
Ah, did not realize that!
| description: (DEPRECATED) The number of times that the redeem code has been used. This field now returns null following a connector update and will be removed in a future release. | ||
| - name: code | ||
| description: The redeemable code that a customer can enter at checkout (e.g., `SUMMER20`). | ||
| - name: created_by_description |
There was a problem hiding this comment.
Same general comment as in src.yml, there appear to be several fields on the to-be-deprecated list that were never documented in the first place.
There was a problem hiding this comment.
See the src yml comment.
| ## Feature Updates | ||
| - Applied removal (or deprecation) of equivalent fields above to the upstream staging models. |
There was a problem hiding this comment.
I would just remove this since there is no new information.
| ## Feature Updates | |
| - Applied removal (or deprecation) of equivalent fields above to the upstream staging models. |
|
|
||
| | Data Model(s) | Change type | Old | New | Notes | | ||
| | ------------- | ----------- | --- | --- | ----- | | ||
| | `shopify__order_lines` | Removed columns | `variant_fulfillment_service`<br>`variant_grams`<br>`variant_weight`<br>`variant_weight_unit`<br>`variant_option_1`<br>`variant_option_2`<br>`variant_option_3` | | [Completes removal of deprecated fields from January updates](https://fivetran.com/docs/connectors/applications/shopify/changelog#january2025). | |
There was a problem hiding this comment.
Do we want to standardize on “remove” or “sunset”? The PR title uses “sunset,” and that’s what we’ve used in other packages, so I’d lean toward “sunset” for consistency--but open to thoughts. Maybe post in the team channel if you want to discuss.
There was a problem hiding this comment.
Good call, let's use sunset, it's the more appropriate term in this case.
…ps://github.com/fivetran/dbt_shopify into schema-change/field-deprecations-and-sunsetting
PR Overview
Package version introduced in this PR:
This PR addresses the following Issue/Feature(s):
Summary of changes:
Submission Checklist
downstream_model_column_countmacro?Changelog